About optimal use of color points of interest for content-based image retrieval

In content-based image retrieval systems, the main approaches based on the query-by-example paradigm involve an approximate search on the whole image, which requires a global description of it. When considering tasks like object recognition or partial queries on particular area, these methods become inadequate and more local characterizations must be employed. In this context, image description based on points of interest appear best adapted. The point characterization which proved reliable is based on invariants to rotation and in particular on combinations of the Hilbert's differential invariants. For gray value images, such a description used to be considered up to third order at least. More recently, generalizations to color images were proposed for stereovision and image retrieval. Some of them propose to consider the invariants only at first order and to enrich the characterization with geometrical constraints for describing spatial relations between points, while others consider higher order invariants and compute some combinations of them to achieve illumination changes invariance. In this report, we discuss the advantages and drawbacks of these different choices, with the aim of proposing an optimal use of color points of interest for content-based image indexing and retrieval

Image Retrieval by Range Query Composition of Region Categories

In this paper, we present an original and new framework to allow complex multiple regions queries based on composition of regions categories. This problematic has merely not been addressed in the litterature even though it corresponds to photo-agency usage scenario. We first present the image segmentation technique specifically designed for image retrieval by regions based on the clustering of Local Distributions of Quantized Colors (LDQC). Then, categories of similar regions are formed automatically by clustering their visual features. The representative regions for each category give to the user an overview of all the regions. They allow him to query the database by specifying different types of regions which should appear in the retrieved images but also types which shouldn't. This enables our system to retrieve images from complex queries of region categories compositio- n. This query mode differs completely from the classic Query-by-Example paradigm since there is no query image. user has a clear idea of what he is looking for. allows him to query Our retrieval system handles region range query by considering regions from the same category but also from neighbor categories. The resulting indexing and retrieval implementation turns out to be simple and very fast even on very complex query compositions and large image database. It was tested on a database of 9,995 images from the Corel image database

Une approche floue du recalage 3D : généricité et robustesse

Projet SYNTIMAn important problem in computer vision is to recover how features extracted from images are connected to either existing models. In this paper, we focus on solving the registration problem, i.e obtaining rigid displacement parameters between several 3D data sets, whether partial or exhaustive. The difficulty of this problem is to obtain a method which is robust to outliers and at the same time accurate. We present a general method performing robust 3D location and fitting based on fuzzy clustering method. The fuzzy set approach is known to its practical efficiency in uncertain environment. To illustrate the advantages of this approach on the registration problem, we show results on synthetic and real 3D data. Moreover, we proposed a tool to compare both 3D data sets.Le recalage 3D est un problème important en vision par ordinateur qui a de nombreuses applications, tant pour obtenir une reconstruction incrémentale complète à partir de primitives 3D extraites de différents points de vue, que pour interpréter une scène reconstruite à partir du modèle des objets qui la composent. Nous nous intéressons ici au recalage rigide entre deux ensembles 3D qui présentent une partie commune. Notre approche est fondée sur l'intégration d'une méthode de classification floue proposée par Dave, qui a été adaptée à notre problème pour obtenir le schéma général d'une méthode de recalage robuste. Quelques résultats sont décrits pour illustrer les avantages de la méthode qui peut en particulier gérer, de manière unifiée, plusieurs solutions recalées. Enfin, nous présentons un outil pour comparer deux ensembles 3D et l'utilisons pour valider notre méthode

Multi-source shared nearest neighbours for multi-modal image clustering

Shared Nearest Neighbours (SNN) techniques are well known to overcome several shortcomings of traditional clustering approaches, notably high dimensionality and metric limitations. However, previous methods were limited to a single information source whereas such methods appear to be very well suited for heterogeneous data, typically in multi-modal contexts. In this paper, we introduce a new multi-source shared neighbours scheme applied to multi-modal image clustering. We first extend existing SNN-based similarity measures to the case of multiple sources and we introduce an original automatic source selection step when building candidate clusters. The key point is that each resulting cluster is built with its own optimal subset of modalities which improves the robustness to noisy or outlier information sources. We experiment our method in the scope of multimodal image search results clustering and show its effectiveness using both synthetic and real data involving different visual and textual information sources and several datasets of the literature.Les techniques basées sur l'information des voisins partagés sont bien connues pour surmonter plusieurs lacunes des méthodes traditionnelles de regroupement, celles liés aussi à la grande dimension et les limites des métriques. Cependant, les méthodes précédentes étaient limitées à une seule source d'information alors qu'elles semblent être très bien adaptées pour des données hétérogènes, généralement dans un contexte multi-modal. Dans cet article, nous introduisons une nouvelle approche multi-source appliquée à une méthode de regroupement basée sur l'information des voisins partagés. Nous avons d'abord étendu les mesures de similarité dans un cas de multiples sources et nous avons introduit une étape originale qui permet la sélection automatique des sources lors de la construction des groupes candidats. L'originalité de la méthode est que chaque groupe qui en résulte est construit avec ses propres sous-ensembles de modalités qui lui sont optimales et qui améliore la robustesse face aux sources d'information bruitées ou aberrantes. Nous avons expérimenté notre méthode dans le cadre de la structuration de résultats de recherche d'images de façon multi-modale et nous avons montré son efficacité en utilisant des données synthétiques et d'autres réelles impliquant différentes sources d'information visuelle et textuelle

Active SVM-based Relevance Feedback with Hybrid Visual and representation

Most of the available image databases have keyword annotations associated with the images, related to the image context or to the semantic interpretation of image content. Keywords and visual features provide complementary information, so using these sources of information together is an advantage in many applications. We address here the challenge of semantic gap reduction, through an active SVM-based relevance feedback method, jointly with a hybrid visual and conceptual content representation and retrieval. We first introduce a new feature vector, based on the keyword annotations available for the images, which makes use of conceptual information extracted from an external ontology and represented by ``core concepts''. We then present two improvements of the SVM-based relevance feedback mechanism: a new active learning selection criterion and the use of specific kernel functions that reduce the sensitivity of the SVM to scale. We evaluate the use of the proposed hybrid feature vector composed of keyword representations and the low level visual features in our SVM-based relevance feedback setting. Experiments show that the use of the keyword-based feature vectors provides a significant improvement in the quality of the results

What's beyond query by example?

Over the last ten years, the crucial problem of information retrieval in multimedia documents has boosted research activities in the field of visual appearance indexing and retrieval by content. In the early research years, the concept of the «query by visual example» (QBVE) has been proposed and shown to be relevant for visual information retrieval. It is obvious that QBVE is not able to satisfy the multiple visual search usage requirements. In this paper, we focus on two major approaches that correspond to two different retrieval paradigms. First, we present the partial visual query that ignores the background of the images and allows a straight user expression on its visual interest without relevance feedback mechanism. The second retrieval paradigm consists in searching for the user mental image when no starting visual example is available. query by logical composition of region categories when a visual thesaurus is generated. This new approach relies on the unpervised generation of a visual thesaurus from which query by logical composition of region categories can be performed. This query paradigm is closely related to that of text retrieval. Mental image search is a challenging and promising issue for retrieval by visual content in the forthcoming years since it allows different rich user expression and interaction modes with the search engine

Adaptive Satellite Images Segmentation by Level Set Multiregion Competition

In this paper, we present an adaptive variational segmentation algorithm of spectral-texture regions in satellite images using level set. Satellite images contain both textured and non-textured regions, so for each region cues of spectral and texture are integrated according to their discrimination power. Motivated by Fisher-Rao's linear discriminant analysis, two region's weights are defined to code respectively the relevance of spectral and texture cues. Therefore, regions with or without texture are processed in the same framework. The obtained segmentation criterion is minimized via curves evolution within an explicit correspondence between the interiors of evolving curves and regions in segmentation. Thus, an unambiguous segmentation to a given arbitrary number of regions is obtained by the multiregion competition algorithm. Experimental results on both natural and satellite images are shown

Large scale visual-based event matching

International audienceOrganizing media according to real-life events is attracting interest in the multimedia community. Event-centric indexing approaches are very promising for discovering more complex relationships between data. In this paper we introduce a new visual-based method for retrieving events in photo collections, typically in the context of User Generated Contents. Given a query event record, represented by a set of photos, our method aims to retrieve other records of the same event, typically generated by distinct users. Similarly to what is done in state-of-the-art object retrieval systems, we propose a two-stage strategy combining an efficient visual indexing model with a spatiotemporal verification re-ranking stage to improve query performance. For efficiency and scalability concerns, we implemented the proposed method according to the MapReduce programming model using Multi-Probe Locality Sensitive Hashing. Experiments were conducted on LastFM-Flickr dataset for distinct scenarios, including event retrieval, automatic annotation and tags suggestion. As one result, our method is able to suggest the correct event tag over 5 suggestions with a 72% success rate

Iterative Search with Local Visual Features for Computer Assisted Plant Identification

To support computer assisted plant species identification in a realistic, uncontrolled picture-taking condition, we put forward an approach relying on local image features. It combines query by example and relevance feedback to support both the localization of potentially interesting image regions and the classification of these regions as representing or not the target species. We show that this approach is successful, and makes prior segmentation unnecessary

Reducing the Redundancy in the Selection of Samples for SVM-based Relevance Feedback

In image retrieval with relevance feedback, the strategy employed by the system for selecting the images presented to the user at every feedback round has a strong effect on the transfer of information between the user and the system. Using SVMs, we put forward a new active learning selection strategy that minimizes redundancy between the images presented to the user and takes into account assumptions that are specific to the retrieval setting. Experiments on several image databases confirm the attractiveness of this selection strategy. We also find that insensitivity to the scale of the data is a desirable property for the SVMs employed as learners in relevance feedback and we show how to obtain such insensitivity by the use of specific kernel functions